Machine learning facilitates complex, statistics-based computations such as speech recognition, image recognition, retrieving information from unorganized text, etc. However, machine learning, and especially probabilistic modeling, can be difficult to apply. To apply machine learning, a user needs to not only design a statistical model, but to also design and implement the right inference procedure to perform inference on the model. There are many different inference algorithms, most of which are conceptually complicated and difficult to implement at scale. Despite the enthusiasm that many people who practice data analysis have for machine learning, this complexity is a barrier to deployment.
Probabilistic programming is a way to simplify the application of machine learning based on Bayesian inference. The key feature of probabilistic programming is separation of concerns: a user specifies what needs to be learned by describing a probabilistic model, while a compiler automatically generates the inference procedure for the described model. In particular, the programmer writes source code that describes one or more probability distributions, and the source code is parsed by a probabilistic programming compiler. The compiler then generates inference code that implements an inference algorithm configured to sample from the one or more described distributions. Using the compiler-generated inference code, the programmer samples from the described distribution(s).
Doing inference on probabilistic programs is computationally intensive and challenging. Most of the algorithms developed to perform inference are conceptually complicated, and it is not clear, especially for non-experts, which algorithm would work best for a given model. It would be beneficial to facilitate selection of an inference algorithm in the context of probabilistic programming.
Furthermore, the amount of data being analyzed via inference code is often voluminous, and a large amount of computation power is required to process such data. In order to produce efficient inference code, it would be beneficial to develop techniques that process large amounts of data efficiently.
The approaches described in this section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued. Therefore, unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section.